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® Method and system for digital Image signatures. 

@ A method and system for embedding signatures within visual images in both digital representation and print 
or film. A signature Is inseparably embedded within the visible image, the signature persisting through image 
transforms that include resizing as well as conversion to print or film and back to digital fonm. Signature pointe 
ate selected from among the pixels of an original image. The pixel values of the signature points and 
surrounding pixels are adjusted by an amount detectable by a digital scanner, "me adjusted signature points 
form a digital signature which is stored for future kientification of subject images derived from the Image. In one 
embodiment, a signature is eml)edded within an Image by locating relative extrema In the continuous space of 
pixel values and selecting the signature points from among the extrema. Preferably, the signature is redurKiantly 
embedded in fte image such th^t any of the redundant representations can be used to identify the signature. 
Wentification of a subject image includes ensuring that the subject image is normalized with respect to the 
original Image or the signed image. Preferably, the normalized subject image is compared with the stored digital 
signature. 
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nSllDiSLOeW ^H,„a a signature into 8 digital image and 

aKdHing a digital subject imago to determ.ne rf was 

R ^oundoi thelnv entton distributed to many users. 

in the computer field, digrtal '"^^^ ^or art digital signaturas haj^ne^ b^app' 
«^ origin of the data. For ^-^J^^^^g^, signatures are ^V^^;^^^. or pl«t«. 

olten desired to resize, rotate o-op or o ^ 
aigital signatures are unacceptaWe for use wnn a g 

§iS!!!ISDL2L!!^^ . „ ^„^res within visual im^es. 

s^tS^^ra^^s^^^^ro^^^^^^ ^ ^ 

^red'^STan^^Si^^ P-ls j^^f ^TuT J^l'- ^""^ ^ "^"^ 

rSSon descrtbed herein -^'^ST^^^^r- ^ 
P^n^jTpSws such as relatwe «*«^ r*,\f^ d^re poW by 
Image by tooating and a data bit is encoded at ^J.^tW embedded in the 

. r^rue-.°2£^L^^^ 

^"^"^ ^ oS^S^ iTSS^^ne Size, rotation, end ^^^J^^^^ Z nance values of 
subiectimage.snonnalu2..^om *«9"^'^..'?i2Tn tiTe original image. The 

^ already nom,allz«l. .mage to match correspondmg subsete ^tn ^ ^.^ 

tte signed image. 

Briof DBScriptiorL glttiaPgglgSg w~r,«pnt of the present invention. 
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Rgure 5 is a digital subject image thai is compared to the image of Rgure 2 according to a preferred 
embodiment of the present invenfion. 



Detailed Description of the Invention 

TTie present invention includes 9 method and system for embedding a signature into an original image 
to create a signed image. A preferred embodiment includes selecting a large number of candidate points in 
the original image and selecting a number of signature points from among the candidate points. The 
signature points are altered slightly to fomn the signature. The signature points are stored for later use in 
TO auditing a subject image to detennine whether the subject image is derived from the signed rmage. 

The signatures are encoded in the visible domain of the image and so become part of the image and 
cannot be detected or removed without prior knowledge of the signature. A key point is that while the 
changes manifested by the signature are too slight to be visible to the human eye. they are easily and 
consistently recognizable by a common digital image scanner, after which the signature is extracted. 
IS interpreted and verified by a software algorithm. ^ 

In contrast to prior art signature mettKxJs used on non-image data, tiie signatures persist through 
significant image transfonnatk>ns that preserve the visible image but may completely change the digital 
data. The specific transforms allowed include resizing the image larger or smaller, rotating the image, 
unrfonnly adjusting cotor. brightness and/or contrast, and limited cropping. Significantly, the signatures 
20 persist through the process of printing the image to paper or film and rescanning it into digital foma. 

Shown in Rgure 1 is a computer system 10 that is used to carry out an embodiment of the present 
invention. The computer system 10 Includes a computer 12 having tiie usual complement of memory and 
logic circuits, a display monitor 14, a keyboard 16. and a mouse 18 or other pointing device. The computer 
system also includes a digital scanner 20 that is used to create a digital image representative of an original 
25 image such as a photograph or painting. Typically, delicate images, such as paintings, are converted to 
print or film before being scanned into digital fonn. In one embodiment a printer 22 is connected to the 
computer 12 to print digital images output from the processor. In addition, digital images can be output in a 
data format to a storage medium 23 such as a floppy disk for displaying later at a renryrte site. Any digital 
display device may be used, such a common computer printer, X-Y plotter, or a display screen. > 
00 An example of the output of the scanner 20 to the computer 1 2 Is a digital image 24 shown in Rgure 2. 
More accurately, the scanner outputs data representative of the digital image and tiie computer causes the 
digital image 24 to be displayed on the display monitor 14. As used herein "digital image" refersvto the 
digital data representative the digital image, tiw digital Image displayed on the monitor or other di^lay 
screen, and the digital image printed tiy the printer 22 or a remote printer. >t 
35 The digital image 24 is depicted using numerous pixels 24 having various pbcel values. In the gray-scale 
image 24 ttie pixel values arB luminance values representing a brightness level varying from black to white. 
In a color image tiie pixels have color values and luminance values, berth of which being pixel values. The 
color values can include the values of any components in a representation of the color by a vector. Rgure 3 
shows digital Image 24A in the fonn of an array of pixels 26. Each pixel is associated with one or more pixel 
40 values, which in the example shown in f=igure 3 are luminance values from 0 to 15. 

The digital image 24 shown in Figure 2 includes ttiousands of pixels. The digital image 24A represented 
in Rgure 3 includes 225 pixels. The Invention preferably is used for images having pixels numbering in ttie 
millions. Therefore, the description herein is necessarily a simplistic discussion of the utility of ti^e invention. 
According to a prefen-ed embodiment of tiie invention numerous candidate points are located within the 
45 original image. Signature points are selected from among the candidate points and are altered to form a 
signature. The signature is a pattern of any number of signahire points. In a preferred embodiment, the 
signature Is a binary number between 16 and 32 bits in length. The signature points may be anywhere 
within an image, but are preferably chosen to be as inconspicuous as possible. Preferably, the number of 
signature points is much greater than the number of bits in a signature. This allows the signature to be 
50 redundantiy encoded in the image. Using a 16 to 32 bit signature, 50-200 signature points are preferable to 
obtain multiple signatures for the image. 

A preferred embodiment of the invention locates candidate points by finding relative nr^axima and 
mituma. collectively referred to as extreme, in tiie image. The extrema represent local extremes of 
luminance or color. Figure 4 shows what is meant by relative extrema. Figure 4 is a graphical representation 
55 of the pixel values of a small portion of a digital image. The vertical axis of the graph shows pixel values 
while ttie horizontal axis shows pixel positions along a single line of tiie digital image. Small undulations in 
pbcel values, indicated at 32. represent portions of the digital image where only small changes in luminance 
or color occur between pixels. A relative maximum 34 represents a pixe l that h as the highest pi xel valu e for 
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a given area of ihe image. Similarly, a relative minimum 36 represents a pixel lhat has the lowest pixel 

"^l^^TZ^^rJX^^^n.^^ P«-n«3 two major reasons. Rr^ ttiey are easily ^^ed by 
siropHS^rr^o^ing. SecorSl. they .towsignatu.^^^ 
6 Or« of the simplest methods to detem,ine relative ortrema is ^^^^J.^^^^'^J^^ZaLr- 
technique. This technique emptoys predatem,ined neighborhoods around each p«el 2e^^-^j'^3 
hood 28 and a large neight»rhood 30. as shown in Figures 2 and 3. In the P'^J^ '"Tt'^ .^^ 

StS,o^ are sU I slmpiicHy. a P^'^'-' --^^ratTre^m^ 

tBt*nioue determines the difference between the average pixel value m the "eignoorn^^ 

,0 Sge piirXe Of the large neighborhood, if the difference Is large compared to the difference for 
surrounding pixels then the first pixel value is a relative maxima or minima. determines 

Usinatiie image of Rgure 3 as an example, the Difference of Averages for the P«e' 26A is a®'^'^ 
as fo" Z. 7al value's within the 3x3 pixel small neighbomood "Pj.^J^^'^S ''''^'^l 

gives an average of 7.67. The pixel vdues within ««5x5 pixel toge n^ghbort,^^ 

« Lding by 25 pixels gives an average of 87B and ^ °f I^^h^Sq f 9^^^ 
in small neighborhood 28G is 10.0; the average In large ^'fl'^'^^/.'^^^ 
Averages for pixel 266 is therefore 0.2. Similar computations on pixels 26B-26F produce the following 
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Based on pixels 26A-2e6. there may be a relative maximum at pixel 26D. inference o^A>^ erf 
159 IS areater than the Difference ofoA>^es for the other examined pixels m *® 
wStliff2BD?s a relative maximum rather than merely a small undulatfer,. -1^ ^rffere^ of Averages 
cS^pared with the Difference of Averages for the pbcete surrour«l.ng It ^^^B^^ 
ISerabl^^Lna within 10% of the image size of any side are not j^^^f ^-^^ ^Sg? n 
orotecTSit loss of signaturB points caused by the practice of cropping the borderarea J^^" 
^prSleTat relative extr^aitiisbare randomly and wid^y spaced a.^ used rather than those that 

^.^gr S^of Averages tec|r^e or other Known techniques, a la^ 

^^?the number dependbig on the pixel density and cortrBSl of *e ^T^^^^^^jf J,"^^^^ 

m of coordin^ which are chosen by keyboard, mouse, or other P"'*"9 f ^tern^. we 
^.SpSlr 12^ be programmed to choose signature points randomly or according to a preprogrammed 

'Tne bH Of binary data is encoded In each signature poW in the '-»S«^V ^^"^^totlt^^ ^ 
and sunounding the point. The image is modified by making a ^^-^^^^^ ^^'Z 
negative adiustmenl in the pixel value at the exact ^^^^^F^^f^J^''^'^'^^^ adjusted 
Di^^ls surrounding each signature point, in approximately a 5 x 5 to 10 x ID O"^- *^^'^^^^ ' 
SSorti^ally tolnsure a S^ntinuous transition to the new vah^e at the ^^r^f^^ ^ ^ 
^^xled in i.es-«naturepointstofom,apattern which isthesign^^^^^ 

. in a preferred embodiment, the signature is a pattern of all of the ^-O^f .^^P^'"^:, ™ ^^3:11 
su««t image, if a statistically signfficant number of potential sigr^re pomte «^ J^^^^^TS 
^n^espondiSg signature points in the signed image, then the ^'^^f'W ^deemed ^^^J^^^"^ 
the signed image. A statistically significant number is somewhat 1^ than 100%. but enough to ce 

ranresentation methods, such as an error-correcting code, may also be usea. ^ 

'Tn^iSr to anew future auditing of images to detem,Ine whether they match ^^^'^"^ '"l^^. 
-signatu^n?-stored-in--a-databaseHn-whichit-i^^ 
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stored by associating the bit value of each signature point together with x-y coordinates of the signature 
point TTie signature may be stored separately or as part of the signed image. The signed image is then 
distributed in digital form. 

As discussed above, the signed image may be transformed and manipulated to form a derived image. 
5 The derived image is derived from the signed image by various transfonnations. such as resizing, rotating, 
adjusting color, brightness and/or contrast, cropping and converting to print or film. TTie derivation may take 
place in multiple steps or processes or may simply be the copying of the signed image directly. 

It is assumed that derivations of these images that an owner wishes to track Include only applications 
which substantially present the resolution and general quality of the image. While a size reduction by 90%, 
10 a significant color alteratton or distinct-pixel-value reduction may destroy the signature, they also reduce the 
images significance and value such that no auditing Is desired. 

In order to audit a subject image according to a pretend embodiment, a user identifies the original 
image of which the subject image is suspected of being a duplicate. For a print or film image, the subject 
image Is scanned to create a digital image file. For a digital image, no scanning is necessary. Tfie subject 
16 digital image is normalized using techniques as descrit^ed below to the same size, and same overall 
brightness, contrast and color profile as the unmodified original image. The subject image is analyzed by 
the method described below to extract the signature, if present, and compare it to any signatures stored for 
that image. 

The normalization process involves a sequence of steps to undo transformations previously made to the 
20 subject image, to return it as close as possible to the resolution and appearance of the original image. It is 
assumed that the subject image has been manipulated and transformed as described above. To align the 
subject image with the original image, a preferred embodiment chooses three or more p>olnts from the 
subject image which correspond to points in the original image. The three or more points of the subject 
image are aligned with the corresponding points in the original image. The points of the subject image not 
25 selected are rotated and resized as necessary to accommodate the alignment of the points selected. 

For example, Rgure 5 shows a digital subject image 38 that is smaller than the original image 24 shown 
in Rgure Z To resize' the subject image, a user pdnts to three points such as tiie inouth 40B, ear 42B and 
eye 44B of the subject image using the mouse 18 or other pointer. Since It Is usually difficult to accurately 
point to a single fMxel, the computer selects the nearest extrema to the pixel ppint^ to by the user. The 
30 user points to the mouth 40A, ear 42A, and eye 44A of the original image. The ccffinputer 12 resizes and 
rotates the subject image as neceissary to ensure tfiat points 408, 42B, and 44B are jibsitioned with respect 
to each other in the same way that points 40A. 42A, and 44A are poationed with nispect to each other in 
ti>e original image. The remaining pbcels are repositioned in proportion to the repositioning of points 40B, 
42B and 44B. By aligning three points the en^re subject image is aligned with the^original Imago without 
36 having to align each pb(et independentiy. 

After the subject image Is aligned, the next step is to normalize the brightness, contrast and/or color of 
the subject im^e. Nonnalizing invoh/es adjusting pixel values Of the subject image to match the value- 
distribution profile of the original image. This is accomplished by a technique analogous to that used to 
align the subject image. A subset of the pixels In the subject image are adjusted to equal corresponding 
40 pixels in the original image. The pbcels not in the subset are adjusted in proportion to the adjustments made 
to the pixels in the subset The pixels of the suhqect image corresponding to the signature points should not 
be among the pixels in the subset Othenyise any signature points in the subject image will be hidden from 
detection when they are adjusted to equal corresponding pixels in the original image. 

In a preferred embodiment, the subset includes the brightest and darkest pbcels of the subject image. 
45 These pixels are adjusted to have luminance values equal to the luminance values of conresponding pixels 
in the original image. To ensure that any signature points can be detected, no signature points should be 
selected during the signature embedding process described atxwe that are among the brightest and 
daricest pixels of the original Image. For example, one could use pixels among the brightest and darkest 3% 
for the adjusting subset, after selecting signature points among less than the brightest and darl^est 5% to 
60 ensure that there is no overlap. 

When the subject image is fully nonrwlized, rt is preferably compared to the original image. One way to 
compare images is to subtract one image from the other. The result of the subtraction is a digital image that 
includes any signature points that were present in the subject image. These signature points, if any, are 
compared to the stored signature points for the signed image. If the signature points do not match, then the 
66 subject image is not an image derived from the signed image, unless the subject image was changed 
substantially from the signed image. 

In an alternative embodiment, the nomnalized subject image is compared directly with the signed image 
instead of subtracting ttie subject imag e from ttie_original iniage. This comparison involves subtracting the 
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. i*«v»« is little or no image resulting from the subtraction then the 

subject image from the signed .mage. « ^l^^^^'y^Zn derWed from the signed image, 
subject image equals to the signed Image, and therefore own o ^ ^, ^ 

T«Zr annate embodiment ir«tead ^ ^^^jt^ ^^^^^^ resolution 

subject image surrounding each potential signature po.m .s '^^^^^ accomplished by selecting each 
and appearance ^ a "rrosp'.nding ^""^ SSi3ons surmundiog each potential ^gnatt^ 

image to be compared directly with a comesponding value of the potential 

IS computed for each poten«al -"OnajKe PoM ^^^^ ^ ^^^^^ 

'r^^ of no..i.ng a. ex^n.a ^^^^ZZ^r^S^:^^^"^ 

ess directed to luminance values, simiter ^^l^^^^Z "biect Image can also be adjusted to 

nom^aB^ng by altering ^^J^J^^^^ it Is not necessary to adjust coto^ 

equal corresponding color values .n an ^^^'J^^ ^ ^ color Image. Color images u«^s 

values in order to encode a signature .n or exfr^ v^^. a digital signature can be encoded in 

having pixel values that include luminance va^u^ fuSance values, color values, or any other 

any JJel values regardless of whether the P^^J^^^^'^Sons may be made more easily to 
tv4 of pixel values. iJJmlnance values are preferred because aiw^^^ 

appended claims. t njifi- 
Claims I ^ 

. .0^^ ^^.r^ - - -'"^ ''^^ 

. ^TSa^u^lcand^-jn^a^ , 

SS^SS^nrtTSm, a .gned image, the adjustod .gna^ 
poinr^^raiues forming a signature for the signed 'mage; and 
storing the agnature fc»r future idontificabon. 
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the extrema. 

values of tb© pixels of the original image, 
the signature points to provide smooth transitions to tnaaojub-B w 
6. The method according to claim 1 . further «)mp W ^ ^a^6^^ points; and 

for the signed image. 

. , ^ o, ^ " - ^ ^ * 

55 comprising: . *rrtm amona the pixels of the original image; 

a signature for the image; and 
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storing the signature for future identification. 

The method according to claim 6, further comprising locating relative extrema in the original image and 
wherein the selecting step includes selecting the signature points from among the extrema. 

The method according to claim 7 wherein the extrema are relative minima or maxima of luminance 
values of the pixels of the original image. 

The method according to claim 6 further comprising: 

selecting a second plurality of signature points from among the candidate points; and 
adjusting the pixel values of the second plurality of signature points to form a redundant signature 
for the signed image. 

The method according to claim 6 wherein the digital image has a border surrounding the image and the 
pixel values adjusted are selected so as not to be within a predetermined distance from the border. 

The method according to claim 6, further comprising adjusting a plurality of pixel values sunrounding 
the signature points to provide smooth transitions to the adjusted pixel values at the signature points. 

The method according to claim 6 wherein the pixel values adjusted are luminance values. 

The method according to claim 6 wherein the pixel values adjusted are color values. 

The method according to claim 6. further comprising analyzing whether a digital subject image 
constitutes or Is derived from a signed image having pwel values that were adjusted to form a signature 
according to claim 6. 

The metiiod according to claim 14 wherein the analyzing step includes normalizing the subject image. 

The method according to claim 15 wherein the nomnalizing step includes aligning tiie subject image 
with the signed image or the original image. . ^j!^- 

17. The method according to daim 16 wherein the aligning step includes selecting three or more pixels in 
the subject image and aligning the three or more peels with corresponding pixels in^^the original or the 
35 agned image. 

ia The method according to claim 15 wherein the pixel values of the subject image and tiie original image 
include luminance values and the nonmaJiang step includes adjusting the luminance values of a subset 
of the pixels in the subject image to equal the luminance values of a corresponding subset of pixels in 
40 the original image. 

19, The method according to daim 14 wherein the analyzing step includes subtracting the subject image 
from the original Image to obtain a resulting image and comparing the resulting image with the stored 
signature. 

45 

20. The mettiod according to claim 14 wherein the analyzing step includes comparing the subject image 
with the signed image. 

21. The method according to claim 14 wherein the analyzing step includes selecting a potential signature 
50 point in the subject image conresponding to a signature point of the signed image and comparing the 

pixel value of the selected point to the pixel value of the conresponding signature point of the signed 
image. 

22, The method according to claim 14 wherein the analyzing step includes selecting a potential signature 
55 point in the subject image corresponding to a signature point of the sighed image, computing an 

average of pixel values of the potential signature point and a plurality of pixels sun^unding the potential 
signature point and comparing the average to the pbtel value of the conresponding signatore point of 
the signed image. 
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form a signature, comprising: . . original image or the signal image, 

corresponding to the pixels of the signature. 

respect to the original image or the signed .mBge. 
with the signal image or the original image. ^ 



signed image. ^ ^ 

the origma! image, 
as digital signature. 

^^^^^^^-^"^-"-^'^'^'^^ 

wHh ttie signed image. 
30. The method according 
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• «,« selection means Includes a pointer operatvely 
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35. The system according to claim 31 wherein the computing device includes means for identifying a 
subject image derived from the signed image. 

3B. The system according to claim 35. further comprising nonmalizing means for normalizing the subject 
5 image with the original Image, or the signed image. 

37. The system according to claim 36 wherein the normalizing means includes a pointer operatively 
connected 1o the display device and the computing device such that a user can select alignment points 
from among the pixels of the subject image displayed on the display device and the computing device 

10 receives the alignment points selected and aligns the subject image with the original image or the 
signed image in response thereto. 

38. The system according to claim 36 wherein the computing device includes comparing means for 
comparing the normalized subject image with the original image or the signed image. 

76 

39. The system accordlrig to claim 36 wherein the computing device includes: 

subject selection means for selecting a potential signature point on the subject Image correspond- 
ing to a signature point of the signed image; 

averaging means for computing an average of the pixel values of the potential signature pdnt and a 
20 plurality of pixels sun'ounding the potential signature point; and 

comparing means for comparing the average to a pixel value of the conresponding signature point 
of the signed image. 
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